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To successfully educate the public about agriculture, food, and natural resources, we must have effective 
educators in both formal and nonformal settings. Specifically, this study, which is a valuable part of a 
larger sequential mixed-method study addressing effective teaching informal and nonformal agricultural 
education, provides direction for future effective teaching research in extension education. Particularly, 
this study assessed 142 behaviors, characteristics, and techniques considered indicative of effective 
teaching, to reduce the number of competencies and identify constructs of effective teaching in extension 
education. A total of 1,470 extension educators from 30 states, surveyed in the fall of 2011, sensed as the 
population for this study. As a result, 63 effective teaching competencies in 11 constructs were identified. 
Psychometric evaluation of the 11 constructs resulted in Cronbach’s alpha coefficients ranging from .82 
to .93, supporting the reliability of the identified constructs. An expert panel then named the constructs, 
many of which aligned with those identified in previous teaching effectiveness research. Implications for 
practice and research resulted from this study, including a proposed three-part framework for assessing 
effective teaching in extension education, which includes self-evaluation, observation-based assessment, 
teaching-related output and/or outcome measures. 
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The National Research Agenda for 
agricultural education noted the need to educate 
and inform the “non-agriculture” public and 
policy makers about agriculture, food, and 
natural resources, must be a top priority for 
agricultural educators (Doerfert, 2011). To be 
successful in that endeavor, agricultural edu¬ 
cators—in both formal and nonformal settings— 
must be effective teachers, capable of cornm.- 
unicating the messages of the agriculture ind¬ 
ustry. The characteristics, behaviors and techni¬ 
ques employed by effective teachers have the 
potential to greatly impact learning and produce 


a subsequent behavior change (Dyer & Osborne, 
1996; Kaiser, McMurdo, & Block Joy, 2007). 

Effective teaching in school-based, or for¬ 
mal, agricultural education environments has 
been the focus of considerable research (e.g., 
Buchanan, 1997; Feldman, 1976; Nicholls, 
2002; Reid & Johnstone, 1999; Rosenshine & 
Furst, 1971; Scheeler, 2008), including in 
agricultural education (Dyer & Osborne, 1996; 
Johnston & Roberts, 2011; Miller, Kahler, & 
Rheault, 1989; Newcomb, McCracken, & 
Warmbrod, 1993; Roberts, Dooley, Harlin, & 
Murphrey, 2007; Roberts & Dyer, 2004). Reid 
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and Johnstone (1999) identified six components 
of quality teaching, including approachability, 
clarity, depth of knowledge, interaction, interest 
and organization. Effective educators must be 
well-prepared, enthusiastic, clear and business¬ 
like, offer variability and involve students in the 
learning process (Etling, 1993). Feldman (1976) 
found stimulating student interest and clarity 
were highly related to effective teaching. Young 
and Shaw (1999) reported effective commu¬ 
nication, a comfortable environment, concern for 
student learning, student motivation, and course 
organization as measures of teacher effect- 
tiveness. Furthermore, Feldman suggested effe¬ 
ctive instructors were knowledgeable about their 
content, prepared and organized for class, and 
were enthusiastic. 

Rosenshine and Furst (1971) studied 
characteristics of effective educators in formal 
settings. Their work suggested the five most 
notable variables associated with effective 
teaching included clarity, variability, enth¬ 
usiasm, task-oriented and/or businesslike 
behavior, and student opportunity to learn 
criterion material (Rosenshine & Furst, 1971). 
Newcomb et al. (1993) identified 13 principles 
of effective teaching believed to impact student 
learning: Students must be motivated to learn, 
reinforced behaviors are most likely to be 
learned, directed learning is more effective than 
undirected learning, students should inquire into 
the subject matter, problem-oriented approaches 
to teaching improves learning, and students learn 
what they practice. 

Although research in formal education 
settings is necessary and beneficial, one could 
argue that education occurring in nonformal 
settings may be even more essential when frying 
to educate the “non-agriculture” public. 
Because both formal and nonformal agricultural 
education programs often stem from 
complementary goals, this has necessitated 
combined educational programming and 
educator training (Shinn & Cheek, 1981; Phipps, 
Osborne, Dyer, & Ball, 2008). Although such 
collaborative efforts are useful, it is also 
necessary to note the differences that exist 
between formal and nonformal settings, and the 
learners in each environment. 

Nonformal education has been defined as a 
category of political and social organizations, an 
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educational strategy, or as a modality of 
teaching and learning (Grandstaff, 1976). 
Historically, Grandstaff (1976) identified the 
desire to associate nonformal education with 
situations where educational content is 
embedded in an activity context. Etling (1993) 
suggested nonformal education is more learner- 
centered than formal education and offers 
options and choices rather than a formal, 
prescribed, sequential curriculum. Additionally, 
Etling (1993) postulated that the freedom of 
nonformal education allows learners to leave 
whenever they lack motivation for learning. 

Gamon, Mohamed, and Trede (1992) 
evaluated the training needs of extension 
personnel. Teaching methods were found to be 
an important component of training for pre¬ 
service and in-service extension personnel 
(Benge, Harder, & Carter, 2011; Haider, Place, 
& Scheer, 2010; Waters & Haskell, 1989). 
Additionally, Cooper and Graham (2001) found 
subject matter competency, teaching decision 
making skills to clients, being familial - with the 
teaching and learning process, experience as a 
teacher, and the ability to train personnel were 
among the core competencies identified as 
necessary for a successful county extension 
agent or county supervisor. Etling (1993) 
suggested effective “educators must emphasize 
those skills, knowledge and attitudes which are 
desired by the learners” (p. 74). Etling (1993) 
also proposed that nonformal educators must be 
flexible and ready to change instruction to meet 
the needs of diverse and evolving dynamics 
exhibited by students. Given these parameters, 
and the somewhat limited research in this area, 
there is great need for further research on 
effective teaching in nonformal settings. 

Frameworks 

The theory of psychometrics provided 
guidance for this study. With the goal of 
developing a model of effective teaching leading 
to a self-assessment and observational instru¬ 
ment, efforts were made to establish psych¬ 
ometric soundness with as few items as possible 
(Ferketich, 1991). Psychometrics allows resea¬ 
rchers to objectively measure concepts through 
indirect means, rather than physical charact¬ 
eristics (Nunnally, 1967). Measurements must 
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include rules for assigning numbers to objects to 
represent quantities of attributes “...to objectify 
the recording of impressions (e.g., rating scales) 
and to objectify the analysis of the results” 
(Nunnally, 1967. p. 486). When proposing a 
new measure (or revising an existing measure), 
it is important to clearly qualify and quantify the 
properties of the concept, thereby providing the 
rules of the measure and the mechanism to 
establish validity and reliability. Empirical 
analyses arc used to create the rules of the 
measure, i.e., legitimate or standardized measure 
of a concept or unitary attribute (Nunnally, 
1967). Measures of several unitary attributes are 
then combined to form an overall objective 
appraisal (Nunnally, 1967). To illustrate this 
concept, one may form an overall objective 
appraisal of an individual’s basic math ability by 
assessing the unitary attributes of his or her 
ability to add, subtract, multiply, and divide— 
the sum of the pieces are then used to assess the 
whole. 

Appraisals are often guided by two 
assessment methods commonly noted in the 
literature, observational assessment and self- 
assessment. Both have their strengths and 
weaknesses, namely the objectivity of the 
assessment protocol. Objectivity is directly 
related to accurate measures, which require 
substantial construct validation. Construct val¬ 
idation begins with establishing functional 
relations among important variables or test items 
(Nunnally, 1967). 

This study focused on behaviors, 
characteristics, and techniques associated with 
effective teaching, largely rooted in a teacher’s 
belief in his or her ability to create desired 
outcomes (Tschannen-Moran & Woolfolk Hoy, 
2001), because “teachers’ efficacy beliefs also 
relate to their behavior in the classroom” (p. 
783). Hence, the development of variables or 
test items was guided by Bandura’s theory of 
self-efficacy (Bandura, 1986). 

Self-efficacy is believed to influence 
thought patterns and emotions that drive actions 
(Bandura, 1986; 1993; 1997). Although teacher 
efficacy may be difficult to measure 
(Tschannen-Moran & Woofholk Hoy, 2001), 
efficacy studies include, but are not limited to, 
references of characteristics, beliefs, behaviors, 
knowledge or competence in specific content 
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areas, and techniques demonstrated by 
efficacious teachers (Allinder, 1994; Bandura, 
1986; 1993; 1997; Berman, McLaughlin, Bass, 
Pauly, & Zellman, 1977; Tschannen-Moran & 
Woolfolk Hoy, 2001). Such characteristics, 
beliefs, behaviors, knowledge, and techniques 
are often referenced when effective teaching is 
described and/or are listed as components of a 
framework of teaching. 

Purpose and Objectives 

Theories must be clearly operationalized 
using constructs before they can be tested 
(Gorsuch, 1983). In some cases, theories arc not 
explicit in operationalizing the associated 
constructs—identifying the components needed 
to measure or test the theory (Gorsuch, 1983). 
In those cases, factor-analytic and psychometric 
analyses provide the mechanisms needed to 
identify the appropriate constructs and the 
associated competencies, before the research 
proceeds (Field, 2009; Gorsuch, 1983). The 
puipose of this study was to identify and 
describe the constructs of effective teaching in 
nonformal settings, through factor-analytic and 
psychometric analyses. The results of this study 
may lead to self-assessment and observational 
instruments for use in future studies. The 
following objectives guided this study: 

1. Assess the factor-analytic and psycho¬ 
metric properties of effective teaching, 
based on the perceptions of extension 
educators. 

2. Using the construct outcomes from 
research objective one, describe 
extension educators’ self-perceived 
ability to perform the competencies 
associated with effective teaching. 

Method 

This study is the quantitative strand of a 
larger sequential mixed-method study, (QUAL 
—> QUAN) as defined by Morse (2003), of 
effective teaching in formal and nonformal 
environments in agricultural education. In 
sequential mixed designs, “mixing occurs across 
chronological phases (QUAL, QUAN) of the 
study; questions or procedures of one strand 
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emerge from or depend on the previous strand” 
(Teddlie & Tashakkori, 2008, p. 151). Mixed- 
method developmental studies in the QUAL —> 
QUAN configuration often identify statements 
or themes through qualitative analysis, followed 
by statistical analyses (Teddlie & Tashakkori, 
2008). 

The preceding qualitative component of the 
larger study asked agricultural educators a series 
of open-ended questions related to effective 
teaching. Researchers examined more than 1,500 
statements through comparative analysis, a 
strategy that can “facilitate the discovery of 
grounded theory” (Glaser & Strauss, 1999, p. 9). 
The comparative analysis yielded 142 unique 
competencies, subsequently used in this study. 
The substantive nature of these competencies 
required further analysis and validation. Thus, 
this study served as the next step in the 
sequential QUAL —> QUAN (Morse, 2003) 
study and sought to develop a closed-ended 
survey instrument (Teddlie & Tashakkori, 2008) 
using the 142 competencies for factor analysis. 

Instrumentation 

A three-section web-based survey instr¬ 
ument was developed and implemented through 
Qualtrics©. In the first section, respondents 
were asked how many years they had been an 
educator, how many hours they teach each week 
(excluding preparation time), and how many 
hours per week they spent preparing to teach. 
The second section included 142 statements 
representing the characteristics, behaviors, and 
techniques related to effective teaching in formal 
and nonformal settings, as identified by 
extension agents and agriculture teachers. Lam 
and Klockars (1982) recommended, “The 
researcher interested in obtaining an interval 
scale may thus be able to eliminate the effort of 
labeling all points on the scale in favor of 
labeling only the endpoints” (p. 321). A five- 
point sliding scale with bipolar anchors (1 = 
Strongly Disagree to 5 = Strongly Agree) was 
used to measure respondents’ agreement with 
the 142 statements. Using the sliding scale, 
respondents indicated their level of agreement to 
the hundredth of one point, providing a more 
finite response than would be obtained by 
simply selecting a whole number. The third 
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section asked respondents to report gender, year 
of birth, highest level of education completed, 
number of hours worked in a typical week, and 
number of hours working with youth 
development activities in a typical week. 

The qualitative strand of the study addressed 
and established content validity for the items 
included. A panel of five experts in extension 
education, instrument development, and/or 
research methods assessed face validity of the 
survey instrument prior to data collection. 
Because an outcome of this study was to 
establish a valid and reliable instrument, both 
validity and reliability were assessed in 
objective one of this study. 

Respondents seldom complete a lengthy 
questionnaire, resulting in item-response bias 
(Dillman, Sinclair, & Clark, 1993; Galesic & 
Bosnjak, 2009). To reduce item-response bias, 
the 142 items included in the second section 
were presented in a random order to each 
respondent, using the randomize function in 
Qualtrics. Additionally, data collected in the 
first section provided a basis of comparison 
between respondents who started the 
questionnaire, but did not finish (n = 125), and 
those who completed the entire questionnaire (n 
- 1,345). Hours typically spent teaching each 
week (excluding preparation time) and hours per 
week typically spent preparing to teach served as 
the dependent variables. 

A multivariate analysis of variance 
(MANOVA) was used to compare variables of 
interest, thus, allowing for “examination of two 
variables while simultaneously controlling for 
the influence of the other variables on each of 
them” (Newton & Rudestam, 1999, p. 137). 
Box’s test of equality of covariance was 
significant {p = .005), indicating that the 
matrices were not the same; however, in large 
samples Box’s test could be significant even 
when covariance matrices are relatively si mi lar 
(Field, 2009). Pillai’s Trace is a powerful test 
that will often detect differences even when 
matrices are different (Field, 2009). Therefore, 
results were interpreted using the Pillai’s Trace 
(V) statistics because of its robustness. The 
result of the MANOVA indicated no significant 
effect of item-response bias on the dependent 
variables V = .002, F( 2, 1,335) = 1.11, p = .330, 
rj p 2 = .002. 
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Population 

Two sampling problems arc associated with 
psychometric development, one related to 
sampling of content, the other related to 
sampling of people (Nunnally, 1967). Sampling 
of people is concerned with the generality of 
findings to populations of persons; whereas, 
sampling of content is related to the generality 
of findings to populations of test items 
(Nunnally, 1967). Because this study was 
exploratory in nature, focus was placed on the 
development of psychological measures, internal 
validity, rather than the ability to infer the results 
to a population. Therefore, the objectives of this 
study were not inferential in nature. 

An explanation of the study was sent to state 
extension service offices or extension educators 
in each state, with a request for names and e- 
mail contacts of extension educators. Sixteen 
states provided lists and another 14 were secured 
from state extension websites, including two 
lists of extension educators associated with 1890 
Land Grant Universities. The accuracy and 
inclusiveness of the lists obtained was unknown. 
It was not reasonably possible to access an 
accurate national frame of extension educators 
or determine the extent of frame error. 

Data included in this study were collected 
from extension educators from 30 U.S. states 
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between September and November 2011. After 
five points of contact (Dillman, Smyth, & 
Christian, 2009), 1,541 responses were received. 
Of those responses, 1,519 indicated that their job 
included teaching formally, informally, or 
nonformally—data from 22 respondents who 
indicated they did not teach were excluded. 
Responses from an additional 49 respondents 
were not included in the analyses related to the 
objectives because more than 50% of the 
questionnaire was incomplete, thus reducing the 
useable sample for this study to 1,470 
respondents. 

The majority (77.6%) of respondents were 
county-, parish-, or borough-level educators, of 
which, 963 possessed the title of Extension 
Agent or Extension Educator; 177 possessed the 
title of Extension Associate or Extension 
Assistant. The remaining 22% of respondents 
consisted of Extension Specialists (n - 203), 
Professors (n - 21), administrators (n = 9), or 
individuals with other appointments (n = 78). 
The survey instrument did not force responses; 
therefore, not all respondents indicated their 
level of service or area of specialization. A 
summary of the respondents’ (n - 1,470) 
characteristics, including years of teaching 
experience and area of specialization are 
included in Table 1. 
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Table 1 


Characteristics of Extension Educator Respondents (n = 1, 470) 


Area of Specialization 


State 

n 

Yrs. Exper. a 

Agriculture 


FCS b 

Youth Dev. c 

Other 

M 

SD 

F 

% d 

/ 

% d 

/ 

% d 

/ 

% d 

AL 

42 

15.09 

10.48 

17 

41.5 

15 

36.6 

14 

34.1 

6 

14.6 

AK 

22 

14.39 

10.95 

3 

16.7 

5 

27.8 

7 

38.9 

9 

50.0 

AZ 

33 

19.66 

10.23 

16 

50.0 

10 

31.3 

14 

43.8 

7 

21.9 

AR 

76 

16.66 

11.33 

31 

43.7 

35 

49.3 

38 

53.5 

5 

7.0 

CA 

26 

16.80 

10.57 

4 

17.4 

2 

8.7 

22 

95.7 

5 

21.7 

CO 

65 

16.49 

10.61 

34 

52.3 

17 

26.2 

30 

46.2 

12 

18.5 

CT 

10 

22.30 

9.92 

3 

30.0 

2 

20.0 

1 

10.0 

4 

40.0 

DE 

11 

17.22 

12.69 

5 

55.6 

1 

11.1 

1 

11.1 

2 

22.2 

FL 

40 

17.35 

10.75 

13 

34.2 

6 

15.8 

28 

73.7 

4 

10.5 

GA 

31 

13.27 

10.74 

12 

37.5 

12 

37.5 

8 

25.0 

0 

0.0 

HI 

13 

22.64 

6.10 

5 

45.5 

5 

45.5 

5 

45.5 

2 

18.2 

IA 

30 

22.21 

10.04 

7 

26.9 

5 

19.2 

15 

57.7 

2 

7.7 

KS 

95 

16.91 

10.44 

40 

43.5 

42 

45.7 

29 

31.5 

7 

7.6 

MD 

42 

14.73 

10.18 

15 

36.6 

17 

41.5 

13 

31.7 

9 

22.0 

MN 

131 

20.40 

10.78 

40 

33.9 

23 

19.5 

23 

19.5 

37 

31.4 

MO 

44 

21.41 

12.57 

11 

25.6 

12 

27.9 

7 

16.3 

13 

30.2 

MT 

35 

13.66 

8.93 

20 

60.6 

10 

30.3 

22 

66.7 

7 

21.2 

NE 

95 

17.51 

11.18 

36 

40.0 

29 

32.2 

41 

45.6 

16 

17.8 

NV 

46 

14.84 

10.47 

10 

22.7 

7 

15.9 

19 

43.2 

22 

50.0 

NH 

39 

18.79 

11.31 

9 

26.5 

11 

32.4 

5 

14.7 

11 

32.4 

NJ 

29 

18.63 

10.94 

6 

24.0 

7 

28.0 

9 

36.0 

6 

24.0 

ND 

45 

16.35 

10.73 

15 

34.9 

16 

37.2 

16 

37.2 

10 

23.3 

OH 

108 

16.11 

9.39 

27 

28.4 

28 

29.5 

31 

32.6 

15 

15.8 

OK 

86 

15.21 

10.15 

30 

34.9 

42 

48.8 

50 

58.1 

8 

9.3 

OR 

68 

16.48 

10.49 

25 

43.1 

17 

29.3 

25 

43.1 

9 

15.5 

SC 

12 

13.45 

8.78 

2 

18.2 

3 

27.3 

11 

100.0 

0 

0.0 

UT 

33 

19.15 

9.37 

17 

51.5 

12 

36.4 

19 

57.6 

5 

15.2 

WA 

84 

16.32 

10.68 

19 

25.0 

25 

32.9 

33 

43.4 

27 

35.5 

WV 

48 

12.93 

9.66 

9 

20.5 

23 

52.3 

20 

45.5 

5 

11.4 

WY 

31 

11.22 

9.78 

12 

40.0 

8 

26.7 

13 

43.3 

8 

26.7 

Total 

1,470 

16.82 

10.64 

493 

36.1 

447 

31.3 

569 

41.3 

273 

19.7 


Note. Not all respondents indicated their years of teaching experience or area of specialization. “Mean 
years of teaching experience. b Family and Consumer Sciences. “Youth Development. d Does not equal 
100% because respondents were asked to “select all that apply” 

The purpose of research objective one was 
Results to assess the factor-analytic and psychometric 
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properties of effective teaching, based on the 
perceptions of extension educators. The 142 
competencies identified in the qualitative strand 
of the larger sequential mixed-method study 
were included in the principal component 
analysis using a varimax rotation. Coefficients 
with an absolute value less than .45 were 
suppressed to eliminate double-loadings. 
Bartlett’s test of sphericity was significant (/; < 
.001) and the Kaiser-Meyer-Olkin (KMO) 
measure of sampling adequacy was .967; values 
above .90 arc considered to be superb (Field, 

Table 2 
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2009). After removing components of less than 
three items and components with Cronbach’s 
alpha coefficients less than .80 (Field, 2009), the 
remaining 63 items composed the 11 -component 
solution that accounted for 67.78% of the total 
variance. The 11-components were then treated 
as independent constructs and served as the 
dependent variables for the study. Eigenvalues, 
percentages of variance, cumulative percentages, 
and Cronbach’s alpha coefficients for each 
construct are reported in Table 2. 


Number of Items, Eigenvalues, Percentages of Variance, Cumulative Percentages for Constructs, and 
Estimates of Reliability 



Items 

Eigenvalue 

% of variance 

Cumulative % 

n 

Cronbach's a 

Construct 1 

11 

8.233 

12.107 

12.107 

1,275 

.928 

Construct 2 

9 

5.376 

7.906 

20.014 

1,321 

.903 

Construct 3 

7 

4.836 

7.112 

27.126 

1,305 

.910 

Construct 4 

7 

4.574 

6.726 

33.852 

1,294 

.881 

Construct 5 

7 

3.845 

5.655 

39.507 

1,283 

.882 

Construct 6 

4 

3.363 

4.946 

44.453 

1,344 

.899 

Construct 7 

5 

3.265 

4.801 

49.254 

1,285 

.820 

Construct 8 

4 

3.258 

4.792 

54.046 

1,364 

.877 

Construct 9 

3 

2.805 

4.125 

58.171 

1,355 

.933 

Construct 10 

3 

2.370 

3.486 

6E657 

1,358 

.883 

Construct 11 

3 

2.364 

3.476 

65.133 

1,352 

.903 


A list of the 11 constructs and corresponding 
items was then distributed to a panel of 10 
experts, who were asked to describe what the 
items in each construct collectively measured. 
Panel members included experts in extension 
education, teaching methods, curriculum devel¬ 
opment, youth development, and program plan¬ 


ning. Once the panel’s feedback was received, 
responses were compiled and evaluated before 
assigning final descriptions to the constructs. 
Construct descriptions and loadings from the 
principal component analysis are reported in 
Table 3. 
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Table 3 

Construct Loadings from Principal Component Analysis with Varimax Rotation _ 

Item _ Loading 

Construct 1: Applied Best Practices in Curriculum Development 

I establish a scope for curriculum. .758 

I provide clear objectives for each lesson. .752 

I create a timeline for curriculum - amount of time for each component. .733 

I use objectives to organize lessons. .709 

I establish a logical sequence for curriculum. .699 

I establish goals that include desired outcomes. .698 

I keep lessons organized to help learners learn information. .685 

I keep lessons organized to help learners retain information. .671 

I follow instructional plans (e.g., lesson or workshop plans). .631 

I present clear objectives. .611 

I use each unit of instruction to introduce the next topic. .554 

Construct 2: Instructional Communication Skills 

I have a strong voice. .741 

I have the ability to be entertaining. .714 

I have a commanding presence. .688 

I vary my voice (I'm not monotone). .679 

I am exciting to watch while teaching. .666 

I have the ability to convey messages at multiple levels. .636 

I am articulate. .620 

I use two-way communication effectively. .516 

I move around the room (not tied to desk or PowerPoint). .495 

Construct 3: Respect for Learners 

I am concerned about learners’ well-being. .753 

I show an apparent interest in learners’ lives. .748 

I am compassionate. .701 

I care about learners. .635 

I show compassion toward learners. .627 

I give attention to all learners. .614 

I am concerned about learners' success. .497 

Construct 4: Professional Approach to Instruction 

I have integrity. .797 

I am trustworthy. .785 

I dress appropriately. .635 

I am honorable. .613 

I honor the individuality of each learner. .565 

I demonstrate humility. .501 

I am responsible. _ .472 

Table 3 Continues 
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Table 3 Continued 

Item _ Loading 

Construct 5: Applied Best Practices in Nonformal Teaching Methods 

I use experiential learning. .638 

I appeal to a variety of learning styles. .561 

I provide a variety of opportunities to learn. .553 

I encourage learner inquiry. .543 

I take advantage of opportunities to learn. .525 

I am flexible with teaching methods. .461 

I take opportunities to improve techniques. .427 

Construct 6: Technical Expertise 

I have experience with the topic. .808 

I am knowledgeable of the topic. .794 

I know how to apply topics to the real world. .688 

I show an evident interest in the topic. .633 

Construct 7: Program Management 

I understand leadership opportunities associated with youth development organizations. .840 

I integrate youth development organizations (e.g. 4-H, FFA, FCCFA. SkillsUSA, etc.). .742 

I communicate with parents/guardians. .673 

I understand how to manage volunteers. .663 

I clearly understand the rules and regulations of the organization. .607 

Construct 8: : Professional Collaboration 

I collaborate with colleagues. .776 

I share resources with colleagues. .761 

I consider advice from colleagues. .722 

I consider constructive criticism from colleagues. .698 

Construct 9: Desire to Teach 

I enjoy teaching. .796 

I want to teach. .795 

I love to teach. .757 

Construct 10: Pragmatic Philosophy of Teaching 

I make real-life connections to the subject matter. .747 

I help learners understand application of the material in the real world. .725 

I provide learners with an opportunity to apply subject matter in a practical way. .714 

Construct 11: Commitment to Fearner Engagement 

I allow learners to ask questions. .728 

I encourage learners to ask questions. .694 

I encourage active participation. _ .684 


Individual items should measure the same 
underlying dimension (Field, 2009), in this case, 
competencies associated with effective teaching. 
Intercorrelations should range from “about .3” to 
no higher than .80 (Field, 2009, p. 648). “If any 
variables have lots of correlations below .3 then 
consider excluding them” (Field, 2009, p. 648). 
Intercorrelations greater than .80 could indicate 
issues related to multicolinearity; thus, those 
items should be removed as well (Field, 2009). 


Even if measuring different aspects of the same 
thing, constructs should correlate (Field, 2009). 
Eight of the 11 constructs had an associated 
correlation score greater than .30 and less than 
.80 (see Table 4). Three bivariate correlation 
scores were less than .30; however, three low 
correlation scores among 55 acceptable bivariate 
correlations were not sufficient cause to remove 
the associated constructs. 
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Table 4 


Bivariate Correlations Between Constructs 


Construct 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

ii 

1 

2 

.574 











3 

.553 

.565 

— 









4 

.546 

.553 

.665 

— 








5 

.663 

.647 

.668 

.597 

— 







6 

.485 

.457 

.538 

.570 

.549 

— 






7 

.489 

.531 

.481 

.512 

.557 

.450 

— 





8 

.400 

.350 

.429 

.384 

.428 

.313 

.158 

— 




9 

.489 

.558 

.582 

.486 

.590 

.379 

.460 

.320 

— 



10 

.505 

.505 

.515 

.489 

.516 

.566 

.440 

.596 

.216 

— 


11 

.440 

.440 

.471 

.565 

.606 

.597 

.488 

.512 

.212 

.507 

— 


The puipose of research objective two was 
to describe extension educators’ self-perceived 
ability to perform behaviors, characteristics, and 
techniques associated with effective teaching. 
Ability scores of the 1,470 extension educators 
in this study are proposed as multi-state 
benchmarks for ability levels in future studies of 
effective teaching. Summated mean and 
standard deviation for each construct arc 
reported in Table 5, by construct and area of 
specialization. Extension educators believed 


they were most efficacious in their Commitment 
to Learner Engagement and Constructivist 
Approach to Instruction; they believed they were 
least efficacious in Applied Best Practices in 
Curriculum Development and Professional 
Collaboration. It is important to note that the 
proposed benchmarks are proposed as a point of 
comparison for future studies and cannot be 
inferred to extension educators beyond the scope 
of this study. 


Table 5 


Construct Benchmark Scores for Extension Educators ’ Ability to Perform Competencies 
(n =1, 470) 


Construct 

M 

SD 

Commitment to Learner Engagement 

4.74 

.392 

Professional Approach to Instruction 

4.65 

.391 

Program Management 

4.53 

.481 

Respect for Learners 

4.50 

.474 

Technical Expertise 

4.50 

.491 

Desire to Teach 

4.49 

.633 

Pragmatic Philosophy of Teaching 

4.46 

.553 

Applied Best Practices in Nonformal Teaching Methods 

4.40 

.481 

Instructional Communication Skills 

4.20 

.535 

Applied Best Practices in Curriculum Development 

3.99 

.595 

Professional Collaboration 

3.91 

.833 


Note. 1 = Strongly Disagree, 5 = Strongly Agree. 
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Conclusions, Implications, and 
Recommendations 

This study resulted in the development of 11 
constructs that described effective teaching in 
extension education. Each construct was 
determined to be valid with acceptable estimates 
of reliability (Cronbach’s a > .80; Field, 2009). 
Prior to this study, benchmarks for effective 
teaching in extension education were not 
obvious in the literature. The construct 
benchmarks presented in this study are not 
proposed as normative data; instead, they arc 
proposed as comparative measures for future 
studies of effective teaching in extension 
education, based on the responses of 1,470 
extension educators from 30 states. 

Many of the 11 constructs identified in the 
objectives of this study confirm or expand the 
findings of previous studies of effective 
teaching—in both formal (Allinder, 1994; 
Bandura, 1986; 1993; 1997; Berman, 

McLaughlin, Bass, Pauly, & Zellman, 1977; 
Rosenshine & Furst, 1971; Tschannen-Moran & 
Woolfolk Hoy, 2001) and nonformal settings 
(Etling, 1993; Feldman, 1976). Specifically, 
extension educators identified the planning and 
organizing of the instruction as meaningful 
characteristics of effective teaching. Newcomb 
et al. (1993) noted that subject matter must 
possess meaning, organization, and structure, 
also indicating the importance of planning and 
organization. 

Considering the nonformal environment of 
this study, similar constructs to Young and Shaw 
(1999) including “effective communication, a 
comfortable learning atmosphere, concern for 
student learning, student motivation, and course 
organization” (Young & Shaw, 1999, p. 682) 
may relate to educator effectiveness, particularly 
as identified in the constructs of Instructional 
Communication Skills and Respect for Learners. 
Newcomb et al. (1993) stated, “Regardless of 
the ages of the persons to be taught, the 
successful teacher systematically makes an 
effort to become knowledgeable about the 
relevant attributes and circumstances of students 
“that impact directly on the students’ 
motivations for instruction” (p. 31). 

Consistently, Cole (1981) also recommended 
that clientele audiences should be considered 

Journal of Agricultural Education 


Developing Metrics for Effective... 

when developing extension teaching methods. 
This study confirms these principles in 
nonformal environments through the attributes 
found in Professional Approach to Instruction 
and Pragmatic Philosophy of Teaching. 

Although items in the constructs of 
Professional Approach to Instruction, Prof¬ 
essional Collaboration, and Desire to Teach 
were found in previous extension literature they 
were more broadly defined as interpersonal 
skills or professionalism (Benge, Harder, & 
Carter, 2011; Harder, Place, and Scheer, 2010). 
The constructs provided here, and the associated 
characteristics, behaviors and techniques may 
guide the training of extension educators to 
correlate these behaviors to effective teaching. 

The constructs of Applied Best Practices in 
Nonformal Teaching Methods and Program 
Management, found to be related to effective 
teaching in this study, verified the importance of 
using experiential learning—in some cases 
through youth organizations—to make learning 
relevant to the participants. These results, and 
other constructs identified in this study, suggest 
many similarities with Roberts and Dyer’s 
(2004) model of effective agriculture teachers 
which included instruction, FFA, SAE, building 
community partnerships, professional gro¬ 
wth/professionalism, program planning, and 
personal qualities. 

When considering the individual items 
associated with each competency, extension 
educators were most self-efficacious in comp¬ 
etencies related to facilitation of programs and 
least self-efficacious in areas related to deve¬ 
loping curriculum and collaborating with coll¬ 
eagues. Given that county-, parish-, or borough- 
level educators often possess a wide variety of 
technical knowledge, it is logical that they 
believed they were self-efficacious in Technical 
Expertise. Further, county-, parish-, or borough- 
level educators often turn to specialists to 
develop or help to develop curriculum; thus, it is 
to be expected that curriculum development is 
the second lowest of the extension educators’ 
perceived abilities. Additionally, working with 
other county-, parish-, or borough-level 
educators to deliver technical programming and 
working with specialists to develop curriculum 
requires the ability to collaborate, which was the 
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lowest of the extension educators’ perceived 
abilities. 

Based on the 11 constructs identified in this 
study through factor analytic procedures, 
extension educators believed they were able to 
perform competencies associated with effective 
teaching, indicating that baseline competency is 
equal to, or above, the means proposed in this 
study. This study did not, however, study the 
importance of the competencies included in the 
11 proposed constructs. Borich (1980) proposed 
three perspectives of competency—knowledge, 
performance, and consequence—to permit a 
more refined evaluation of educator need for 
professional development; all of which take into 
account the importance of each competency. 
Therefore, it is recommended that future studies 
use the 63 competencies identified in this study 
and the Borich (1980) needs assessment model 
to conduct needs assessments of extension 
educators. 

Further, as suggested by Rosenshine and 
Furst (1971), observation protocols should be 
developed to complement the self-assessment 
protocol developed in this study. The 
observation protocols would likely help 
extension supervisors and administrators to 
provide effective feedback to extension 
educators. When self-assessment and 
observation-based assessment of effective 
teaching are considered collectively, the results 
are more likely to be valid; however, the 
validation of effective teaching is provided by 
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measures of learner outcomes (Borich, 1979), 
i.e., direct improvement in performance or 
adoption of behavior resulting in achievement. 
Although some form of these measures may be 
widely available in formal settings (e.g., end of 
course exams, state-mandated standardized 
exams), measures of learner outcomes based on 
extension education are less prevalent in 
extension’s nonformal settings. Thus, state-level 
extension administrators and specialists should 
work closely with county-, parish-, or borough- 
level extension educators to identify or develop 
standardized output or outcome measures on a 
state-by-state basis. Additionally, if proactively 
approached by multiple states, the Plan of Work 
mandated by the U.S. Department of Agriculture 
under the Agricultural Research, Extension, and 
Education Reform Act of 1998, may provide a 
stalling point to develop widely used measures 
to correlate with self-assessments and 
observation-based assessments. 

Collectively, self-assessed, observation- 
based, and output/outcome-based measures of 
effective teaching should be considered as a new 
model (see Figure 1) to assess effective teaching 
in extension education. This proposed 
triangulation of teacher effectiveness in 
extension education would provide more 
credible and valid evaluation data for extension 
educators and state administrators, and could 
provide long-term improvement in extension 
program delivery for local stakeholders. 
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Figure 1. A three-component model to assess effective teaching in extension education. 
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